Telegram Group & Telegram Channel



tg-me.com/david_random/564
Create:
Last Update:

8-9 t/s并没有完全发挥8060S的潜力,llama.cpp的llama-server有一个小问题导致server配speculative decoding时性能欠佳(与具体硬件无关):https://github.com/ggml-org/llama.cpp/issues/12968

动手简单修一下这个问题之后Qwen 2.5 72B iq4_xs + 1.5B draft在acceptance rate理想时可达到10-12 t/s左右
https://github.com/hjc4869/llama.cpp/commit/0b32f64ffbe973e99e0dc7097be31d4d966d476e

BY David's random thoughts





Share with your friend now:
tg-me.com/david_random/564

View MORE
Open in Telegram


David& 39;s random thoughts Telegram | DID YOU KNOW?

Date: |

Launched in 2013, Telegram allows users to broadcast messages to a following via “channels”, or create public and private groups that are simple for others to access. Users can also send and receive large data files, including text and zip files, directly via the app.The platform said it has more than 500m active users, and topped 1bn downloads in August, according to data from SensorTower.

Why Telegram?

Telegram has no known backdoors and, even though it is come in for criticism for using proprietary encryption methods instead of open-source ones, those have yet to be compromised. While no messaging app can guarantee a 100% impermeable defense against determined attackers, Telegram is vulnerabilities are few and either theoretical or based on spoof files fooling users into actively enabling an attack.

David& 39;s random thoughts from us


Telegram David's random thoughts
FROM USA